Papers with judgment model
Can You Really Trust Code Copilot? Evaluating Large Language Models from a Code Security Perspective (2025.acl-long)
Copied to clipboard
| Challenge: | Existing code security benchmarks focus on one task and paradigm, such as code completion and generation, without comprehensive assessment across dimensions like secure code generation, vulnerability repair and discrimination. |
| Approach: | They propose a multi-task benchmark for comprehensive evaluation of LLM code security . they also propose VC-Judge, an improved judgment model that aligns closely with human experts . |
| Outcome: | The proposed model can evaluate LLM-generated programs for vulnerabilities in a more efficient and reliable way. |